import Tabs from "@theme/Tabs";
import TabItem from "@theme/TabItem";

# Writing Tests

Let's assume we'd like to test a software that takes the username of a student
and returns basic information about them including their name, date of birth,
and GPA.

<Tabs groupId="sdk" queryString>
  <TabItem value="python" label="Python">

```py
def find_student(username: str) -> Student:
    # ...
```

  </TabItem>
  <TabItem value="cpp" label="C++">

```cpp
Student find_student(const std::string& username);
```

  </TabItem>
  <TabItem value="js" label="JavaScript">

```ts
export async function find_student(username: string): Student;
```

  </TabItem>
  <TabItem value="java" label="Java">

```java
public static Student findStudent(final String username);
```

  </TabItem>
</Tabs>

Where `Student` has the following properties:

<Tabs groupId="sdk" queryString>
  <TabItem value="python" label="Python">

```py
@dataclass
class Student:
    username: str
    fullname: str
    dob: datetime.date
    gpa: float
```

  </TabItem>
  <TabItem value="cpp" label="C++">

```cpp
struct Student {
  std::string username;
  std::string fullname;
  Date dob;
  float gpa;
};
```

  </TabItem>
  <TabItem value="js" label="JavaScript">

```ts
interface Student {
  username: string;
  fullname: string;
  dob: Date;
  gpa: number;
}
```

  </TabItem>
  <TabItem value="java" label="Java">

```java
import java.time.LocalDate;

public class Student {
    public String username;
    public String fullname;
    public LocalDate dob;
    public double gpa;
}
```

  </TabItem>
</Tabs>

Here's a Touca test we can write for our code under test:

<Tabs groupId="sdk" queryString>
  <TabItem value="python" label="Python">

```py
import touca
from students import find_student

@touca.workflow
def students_test(username: str):
    with touca.scoped_timer("find_student"):
        student = find_student(username)
    touca.assume("username", student.username)
    touca.check("fullname", student.fullname)
    touca.check("birth_date", student.dob)
    touca.check("gpa", student.gpa)
```

  </TabItem>
  <TabItem value="cpp" label="C++">

```cpp
#include "students.hpp"
#include "students_types.hpp"
#include "touca/touca.hpp"

int main(int argc, char* argv[]) {
  touca::workflow("find_student", [](const std::string& username) {
    const auto& student = find_student(username);
    touca::assume("username", student.username);
    touca::check("fullname", student.fullname);
    touca::check("birth_date", student.dob);
    touca::check("gpa", student.gpa);
    touca::add_metric("external_source", 1500);
  });
  return touca::run(argc, argv);
}
```

  </TabItem>
  <TabItem value="js" label="JavaScript">

```ts
import { touca } from "@touca/node";
import { find_student } from "./students";

touca.workflow("students_test", async (username: string) => {
  touca.start_timer("find_student");
  const student = await find_student(username);
  touca.stop_timer("find_student");
  touca.assume("username", student.username);
  touca.check("fullname", student.fullname);
  touca.check("birth_date", student.dob);
  touca.check("gpa", student.gpa);
  touca.add_metric("external_source", 1500);
});

touca.run();
```

  </TabItem>
  <TabItem value="java" label="Java">

```java
import io.touca.Touca;

public final class StudentsTest {

  @Touca.Workflow
  public void findStudent(final String username) {
    Touca.startTimer("find_student");
    Student student = Students.findStudent(username);
    Touca.stopTimer("find_student");
    Touca.assume("username", student.username);
    Touca.check("fullname", student.fullname);
    Touca.check("birth_date", student.dob);
    Touca.check("gpa", student.gpa);
    Touca.addMetric("external_source", 1500);
  }

  public static void main(String[] args) {
    Touca.run(StudentsTest.class, args);
  }
}
```

  </TabItem>
</Tabs>

With the following general pattern:

<Tabs groupId="sdk" queryString>
  <TabItem value="python" label="Python">

```py
import touca

@touca.workflow
def name_of_suite(testcase: str):
    # your code goes here
```

  </TabItem>
  <TabItem value="cpp" label="C++">

```cpp
#include "touca/touca.hpp"
#include <string>

int main(int argc, char* argv[]) {
  touca::workflow("workflow_name", [](const std::string& testcase) {
  // your code goes here
  });
  return touca::run(argc, argv);
}
```

  </TabItem>
  <TabItem value="js" label="JavaScript">

```ts
import { touca } from "@touca/node";

touca.workflow("name_of_suite", (testcase: string) => {
  // your code goes here
});

touca.run();
```

  </TabItem>
  <TabItem value="java" label="Java">

```java
import io.touca.Touca;

public final class ExampleTest {

  @Touca.Workflow
  public void workflowName(final String testcase) {
    // your code goes here
  }

  public static void main(String[] args) {
    Touca.run(ExampleTest.class, args);
  }
}
```

  </TabItem>
</Tabs>

The code we insert as our workflow under test generally performs the following
operations:

1.  Map a given testcase name to its corresponding input.
2.  Call the code under test with that input.
3.  Describe the behavior and performance of the code under test.

You can define any number of test workflows and run one or all of them as part
of the same test. The test runner will execute the code under test with the test
cases for each workflow and submits the test results to the corresponding suite.
You can use command line option `--filter` to limit the test run to any given
workflow.

<Tabs groupId="sdk" queryString>
<TabItem value="python" label="Python">

```py
# highlight-next-line
@touca.workflow(testcases=["alice", "bob", "charlie"])
def students_test(username: str):
    student = code_under_test.find_student(username)
    touca.check("gpa", student.gpa)

# highlight-next-line
@touca.workflow(testcases=["banana", "orange"])
def fruits_test(name: str):
    fruit = code_under_test.find_color(name)
    touca.check("color", fruit.color)
```

</TabItem>
<TabItem value="cpp" label="C++">

```cpp
// highlight-next-line
touca::workflow(
    "students_test",
    [](const std::string& username) {
      const auto& student = find_student(username);
      touca::check("gpa", student.gpa);
    },
    [](touca::WorkflowOptions& w) {
      w.testcases = {"alice", "bob", "charlie"};
    });

// highlight-next-line
touca::workflow(
    "fruits_test",
    [](const std::string& name) {
      const auto& fruit = find_fruit(name);
      touca::check("color", fruit.color);
    },
    [](touca::WorkflowOptions& w) {
      w.testcases = {"banana", "orange"};
    });
```

</TabItem>
<TabItem value="js" label="JavaScript">

```ts
// highlight-next-line
touca.workflow(
  "students_test",
  async (username: string) => {
    const student = await find_student(username);
    touca.check("gpa", student.gpa);
  },
  { testcases: ["alice", "bob", "charlie"] }
);

// highlight-next-line
touca.workflow(
  "fruits_test",
  (name: string) => {
    const fruit = find_fruit(name);
    touca.check("color", fruit.color);
  },
  { testcases: ["banana", "orange"] }
);
```

</TabItem>
<TabItem value="java" label="Java">

```java
public final class ToucaTests {

  // highlight-next-line
  @Touca.Workflow
  public void findStudent(final String username) {
    Student student = Students.findStudent(username);
    Touca.check("gpa", student.gpa);
  }

  // highlight-next-line
  @Touca.Workflow
  public void findFruit(final String name) {
    Fruit fruit = Fruits.findFruit(name);
    Touca.check("color", fruit.color);
  }

  public static void main(String[] args) {
    Touca.setWorkflowOptions("findStudent", x -> {
      x.testcases = new String[] { "alice", "bob", "charlie" };
    });
    Touca.setWorkflowOptions("findFruit", x -> {
      x.testcases = new String[] { "banana", "orange" };
    });
    Touca.run(ToucaTests.class, args);
  }
}
```

</TabItem>
</Tabs>

## Describing the Behavior

For any given username, we can call our code under test and capture the
important properties of its output that we expect to remain the same in future
versions of our software.

We can start small and capture the entire returned object as a Touca result:

<Tabs groupId="sdk" queryString>
  <TabItem value="python" label="Python">

```py
touca.check("student", student)
```

  </TabItem>
  <TabItem value="cpp" label="C++">

```cpp
touca::check("student", student);
```

  </TabItem>
  <TabItem value="js" label="JavaScript">

```ts
touca.check("student", student);
```

  </TabItem>
  <TabItem value="java" label="Java">

```java
Touca.check("student", student);
```

  </TabItem>
</Tabs>

What if we decided to add a field to the return value of the function that
reported whether the profile was fetched from the cache? Since this information
may change every time we run our tests, we can choose to capture different
fields as separate entities.

<Tabs groupId="sdk" queryString>
  <TabItem value="python" label="Python">

```py
touca.assume("username", student.username)
touca.check("fullname", student.fullname)
touca.check("birth_date", student.dob)
touca.check("gpa", student.gpa)
```

  </TabItem>
  <TabItem value="cpp" label="C++">

```cpp
touca::assume("username", student.username);
touca::check("fullname", student.fullname);
touca::check("birth_date", student.dob);
touca::check("gpa", student.gpa);
```

  </TabItem>
  <TabItem value="js" label="JavaScript">

```ts
touca.assume("username", student.username);
touca.check("fullname", student.fullname);
touca.check("birth_date", student.dob);
touca.check("gpa", student.gpa);
```

  </TabItem>
  <TabItem value="java" label="Java">

```java
Touca.assume("username", student.username);
Touca.check("fullname", student.fullname);
Touca.check("birth_date", student.dob);
Touca.check("gpa", student.gpa);
```

  </TabItem>
</Tabs>

This approach allows Touca to report differences in a more helpful format,
providing analytics for different fields. If we changed the implementation to
always capitalize student names, we could better visualize the differences to
make sure that only the value associated with key `fullname` changes across our
test cases.

Note that we used Touca function `assume` to track the `username`. Touca does
not visualize the values captured as assumption unless they are different.

We can capture the value of any number of variables, including the ones that are
not exposed by the interface of our code under test. In our example, let us
imagine that our software calculates GPA of students based on their courses.

If we are just relying on the output of our function, it may be difficult to
trace a reported difference in GPA to its root cause. Assuming that the courses
enrolled by a student are not expected to change, we can track them without
redesigning our API:

<Tabs groupId="sdk" queryString>
  <TabItem value="python" label="Python">

```py
def calculate_gpa(courses: List[Course]):
    touca.check("courses", courses)
    return sum(k.grade for k in courses) / len(courses) if courses else 0
```

  </TabItem>
  <TabItem value="cpp" label="C++">

```cpp
float calculate_gpa(const std::vector<Course>& courses) {
  touca::check("courses", courses);
  const auto& sum = std::accumulate(courses.begin(), courses.end(), 0.0f,
      [](const float sum, const Course& course) {
        return sum + course.grade;
      });
  return courses.empty() ? 0.0f : sum / courses.size();
}
```

  </TabItem>
  <TabItem value="js" label="JavaScript">

```ts
function calculate_gpa(courses: Course[]): number {
  touca.check("courses", courses);
  return courses.reduce((sum, v) => sum + v.grade, 0) / courses.length;
}
```

  </TabItem>
  <TabItem value="java" label="Java">

```java
private static double calculateGPA(final Course[] courses) {
  Touca.check("courses", courses);
  double sum = Arrays.asList(courses).stream().mapToDouble(item -> item.grade).sum();
  return courses.length == 0 ? sum / courses.length : 0.0;
}
```

  </TabItem>
</Tabs>

Touca data capturing functions remain no-op in production environments. They are
only activated when running in the context of a Touca test workflow.

## Describing the Performance

Just as we can capture values of variables to describe the behavior of different
parts of our software, we can capture the runtime of different functions to
describe their performance. Touca can notify us when future changes to our
implementation result in significant changes in the measured runtime values.

<Tabs groupId="sdk" queryString>
  <TabItem value="python" label="Python">

```py
touca.start_timer("find_student")
student = find_student(username)
touca.stop_timer("find_student")
```

The two functions `start_timer` and `stop_timer` provide fine-grained control
for runtime measurement. If they feel too verbose, we can opt to use
`scoped_timer` instead:

```py
with touca.scoped_timer("find_student"):
    student = find_student(username)
```

  </TabItem>
  <TabItem value="cpp" label="C++">

```cpp
touca::start_timer("find_student");
const auto& student = find_student(username);
touca::stop_timer("find_student");
```

The two functions `start_timer` and `stop_timer` provide fine-grained control
for runtime measurement. If they feel too verbose, we can opt to use
`scoped_timer` as an alternatives:

```cpp
Student find_student(const std::string& username) {
  TOUCA_SCOPED_TIMER;
  // implementation
}
```

We can also measure the lifetime of a scoped variable:

```cpp
touca::scoped_timer timer("find_student");
```

  </TabItem>
  <TabItem value="js" label="JavaScript">

```ts
touca.start_timer("find_student");
const student = find_student(username);
touca.stop_timer("find_student");
```

The two functions `start_timer` and `stop_timer` provide fine-grained control
for runtime measurement. If they feel too verbose, we can opt to use
`scoped_timer` as an alternatives:

```ts
const student = await touca.scoped_timer("find_student", () =>
  find_student(username)
);
```

  </TabItem>
  <TabItem value="java" label="Java">

```java
Touca.startTimer("find_student");
Student student = Students.findStudent(username);
Touca.stopTimer("find_student");
```

The two functions `startTimer` and `stopTimer` provide fine-grained control for
runtime measurement. If they feel too verbose, we can opt to use `scopedTimer`
as an alternatives:

```java
Touca.scopedTimer("find_student", () -> {
    student = Students.findStudent(username);
});
```

Alternatively, we could use `io.touca.ScopedTimer` in a `try-with-resources`
statement:

```java
try (ScopedTimer timer = ScopedTimer("find_student")) {
    Student student = Students.findStudent(username);
}
```

  </TabItem>
</Tabs>

It is also possible to add measurements obtained by other performance
benchmarking tools.

<Tabs groupId="sdk" queryString>
  <TabItem value="python" label="Python">

```py
touca.add_metric("external_source", 1500)
```

  </TabItem>
  <TabItem value="cpp" label="C++">

```cpp
touca::add_metric("external_source", 1500);
```

  </TabItem>
  <TabItem value="js" label="JavaScript">

```ts
touca.add_metric("external_source", 1500);
```

  </TabItem>
  <TabItem value="java" label="Java">

```java
Touca.addMetric("external_source", 1500);
```

  </TabItem>
</Tabs>

In addition to these data capturing functions, the test framework automatically
tracks the wall-clock runtime of every test case and reports it to the Touca
server.

Like other data capturing functions, we can use Touca performance logging
functions in production code, to track runtime of internal functions for
different test cases. The functions introduced above remain no-op in production
environments.

## Running the test

We can run our test from the command line:

<Tabs groupId="sdk" queryString>
  <TabItem value="python" label="Python">

```bash
touca config set api-key=<TOUCA_API_KEY>
touca config set api-url=<TOUCA_API_URL>
touca test --testcase alice bob charlie
```

  </TabItem>
  <TabItem value="cpp" label="C++">

```bash
export TOUCA_API_KEY="your-api-key"
export TOUCA_API_URL="your-api-url"
./local/dist/bin/example_cpp_main_api --testcase alice,bob,charlie
```

  </TabItem>
  <TabItem value="js" label="JavaScript">

```bash
export TOUCA_API_KEY="your-api-key"
export TOUCA_API_URL="your-api-url"
node ./students_test.js --testcase alice bob charlie
```

  </TabItem>
  <TabItem value="java" label="Java">

```bash
export TOUCA_API_KEY="your-api-key"
export TOUCA_API_URL="your-api-url"
gradle runExampleMain --args='--testcase alice bob charlie'
```

  </TabItem>
</Tabs>

Touca SDK captures the `Student` object with all its properties and submits that
information to the Touca server. We can check this output on the web app but we
can also ask the SDK to generate a JSON result file for us:

<Tabs groupId="sdk" queryString>
  <TabItem value="python" label="Python">

```bash
touca test --save-as-json
```

  </TabItem>
  <TabItem value="cpp" label="C++">

```bash
./local/dist/bin/example_cpp_main_api --save-as-json
```

  </TabItem>
  <TabItem value="js" label="JavaScript">

```bash
node ./students_test.js --save-as-json
```

  </TabItem>
  <TabItem value="java" label="Java">

```bash
gradle runExampleMain --args='--save-as-json'
```

  </TabItem>
</Tabs>

You can use `--help` to learn about available command line options.

Notice that we are not specifying the list of test cases anymore. When they are
not explicitly provided, the SDK fetches this list from the Touca server.
